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Abstract 

Let Xi, . . . ,Xn S R™, n < TV, be independent centered random vectors with log-concave distribution 
and with the identity as covariance matrix. We show that with overwhelming probability one has 



sup 

xeS n ~ 



1 / — 

±52(\(X t ,x)? -V\(Xi, X )\ a )\<cJ±, 



where C is an absolute positive constant. This result is valid in a more general framework when the linear 
forms {{Xi,x)) i<N xeS m-i and the Euclidean norms (|-Xi|/v^)i<iV exhibit uniformly a sub-exponential 
decay. As a consequence, if A denotes the random matrix with columns (Xi), then with overwhelming 
probability, the extremal singular values Amin and A max of AA T satisfy the inequalities 1 — C\/^" < 
< A y < 1 + C<J~jj which is a quantitative version of Bai-Yin theorem [4] known for random 
matrices with i.i.d. entries. 

Let X £ 1" be a centered random vector whose covariance matrix is the identity and Xi, Ajv be 
independent copies of X. Let A be a random n x N matrix whose columns are (Xi). By A m j n (resp. A max ) 
we denote the smallest (resp. the largest) singular number of the matrix of empirical covariance AA T . In 
the study of the local regime in the random matrix theory of particular interest is the limit behavior of 
extremal values of the spectrum of AA T . In the case of Wishart Ensemble when the coordinates of X are 
independent, the Bai-Yin theorem [4] establishes the convergence of A m i n /iV and A max /iV when n, N — > oo 
and n/N — ¥ f3 € (0,1), under the assumption of a finite fourth moment. In this note we study the asymptotic 
non-limit behavior (also called "non-asymptotic" in Statistics) i.e. we look for sharp upper and lower bounds 
for singular values in terms of n and N, when n < N are sufficiently large. For example, for Gaussian 
matrices it is known that singular values satisfy inequalities 

^ - ^min . ^max - - ™ I ^ 
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with probability close to 1. We obtain the same estimates for large class of random matrices, which in 
particular do not require that entries of the matrix are independent or that Xj's are identically distributed. 
Note that the natural question about convergence of singular values in such a case is still open (see [2] for 
the case of Xi having uniform distribution on a rescaled ball). 

The natural scalar product and Euclidean norm on R™ are denoted by ( ■, •) and | • |. We also denote by 
the same notation | • | the cardinality of a set. By C, C±, c etc. we will denote absolute positive constants. 

Let Xi, . . . ,Xjy be a sequence of random vectors in R n (not necessarily identically distributed). We say 
that it is uniformly if for some ip > 0, 

sup sup || \{Xi,y)\ |Ux <i>, (2) 

where for a random variable Y € R, ||^||^i = hif {C > ; Eexp (|Y|/C) < 2} . We say that it satisfies the 
boundedness condition with constant K (for some K > 1) if 

p(max|X l |/V^>ifmax{l,(A^/n) 1/4 } ) < exp (-- y/n) • (3) 

The main result of this note is the following theorem. 

Theorem 1 Let n < N be positive integers and ip,K > 1. Let X\, . . . , X/v be independent random vectors 
in R n satisfying $2$) and (0). Then with probability at least 1 — 2 exp (— Cy/n) one has 

1 N I — 

su p J^EO^)! 2 -^'^ 2 ) <C{^ + Kf J-. 

Remarks. 1. Theorem [T] improves estimates obtained in [T] for log-concave isotropic vectors. There, we 
considered essentially the case of N proportional to n, which was sufficient to answer the question of Kannan, 
Lovasz and Simonovits 0, however, for bigger N, the results were off by a logarithmic factor. The theorem 
above removes this factor completely leading to the best possible estimate for an arbitrary N, that is to an 
estimate of the same order as in the Gaussian case. 

2. In the case N < n Theorem [5] below together with assumptions @ and © immediately implies that the 
norm of the matrix A with columns X\ , . . . , Xjy satisfies 

|| A|| < C(yj + K) 

with probability at least 1 — 2 exp (—c-s/n). This in turn implies that A max < C {ip + K) 2 n and that 

1 N 

sup -^(KX^-EKX^)! 2 ) <C{^ + Kf ^ 



i=l 



with the same probability. 

As a consequence of Theorem [T] we obtain the following quantitative version of Bai-Yin theorem [2] known 
for random matrices with i.i.d. entries. 

Corollary 1 Let A be a random n x N matrix, whose columns X\ , . . . , Xn are isotropic random vectors 
satisfying the assumptions of Theorem\]\ Then with probability at least 1 — 2exp(— c^/n), 

l-C^ + KfJ^<^<^<l + C(^ + KfM 
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To emphasize the strength of the above results we observe that conditions © and §3§ are valid for many 
classes of distributions. 



Example 1 Random vectors uniformly distributed on the Euclidean ball of radius Ky/n clearly satisfy 
They also satisfy © with ip = CK. 

Example 2 Log-concave isotropic random vectors in R n . Recall that a random vector is isotropic if its 
covariance matrix is the identity and it is log-concave if its distribution has a log-concave density. Such 
vectors satisfy (|2|) and <j3j) for appropriate absolute constants ip and K. The boundedness condition follows 
from Paouris' theorem ([7]) and is explicitly written e.g., in pQ, Lemma 3.1. We would like to remark that 
a version of Theorem Q] with a weaker probability estimate was proved by Aubrun in the case of isotropic 
log-concave random vectors under an additional assumption of unconditionality (see [3]). 

Example 3 Any isotropic random vectors (X;)j<jv in l^ n , satisfying the Poincare inequality with constant L, 
i.e. such that Var(/(X;)) < L 2 E|V/(Xi))| 2 for all compactly supported smooth functions, satisfy ((2|) with 
ip — CL and (j3|) with K = CL. The question from [5] whether all log-concave isotropic random vectors 
satisfy the Poincare inequality with an absolute constant is one of the major open problems in the theory of 
log-concave measures. 

The proof of Theorem [T] is close to arguments in Section 4.3 of pQ, however it uses a choice of parameters 
more appropriate for the case considered here, and a new approximation argument. We need additional 
notations. Let 1 < m < N. By U m we denote the subset of all vectors in S^ -1 having at most m non-zero 
coordinates. For annxJV matrix A we let 



The main technical tool is the following result which is the "in particular" part of Theorem 3.13 from PQ in 
which one needs to adjust corresponding constants and to take a union bound. 

Theorem 2 Let X\, . . . ,Xn be as in Theorem^ let A be a random n x N matrix whose columns are the 
Xi 's. Then for every t > 1 one has 



A m = sup \Az\. 

zeu m 



(4) 



P 3m A m > 



Cipt max{-\/TOln , y/n} + 6 max |JQ| j < exp (— t-\/nj 




Proof of Theorem [H For x e S' 



<n— 1 



set 



i=l 



Let B > be a parameter which we specify later and observe that 




sup S(x)< sup ( - Y^((\(X t ,x)\AB) 2 -E(\(X^x)\ A B) 2 



+ nT, (\( X *> x )\ 3 - B2 ) M\(x^)\>b } + (\(X t ,x)\ 2 -B 2 ) l { \ {Xi , x) \> B} ). 




3 



We denote the summands under the supremum by Si(x), S2(x), and Sa(x), respectively. 

Estimate for 5 X : Given x € S 71 ' 1 and i < N let Z t = ^(x) = (|(X;,x)| A B) 2 -E(|(JQ,x)| A B) 2 . Then 
\ < -S 2 - Moreover, since 

Var{Zi) <E{\(X h x)\ AB) 4 <E\(Xi,x)\ 4 <Ct^, 
we observe that a 2 = jj J2iLi Var(Zj) < Clip 4 . Thus, by Bernstein's inequality 

9 2 N 



'(Si(x)>8) 



(^f>^) " eXp 



2(CiV' 4 + B 2 9 /3) 



It is well known that 5 1 ™ 1 admits a (l/3)-net A/" in the Euclidean metric such that \N\ < 7™. Then by the 
union bound we obtain that if 

9 2 N > 8CiV/nhi7 and 6N > (8/3)B 2 n In 7 (5) 

then 

(\ / # 2 iV \ / 9 2 N \ 

sup SlW > •) < exp (nln7- ^ - < exp (- ^-^ j . (6) 

Estimates for S2 and By Holder's inequality and ((2]) we have, for some absolute constant C2 > 1, 



1 N 

sup S 3 (x)<-V sup |KX i ,x)|||P(KX i ,x)| >B) 1/2 <C 2 V 2 exp(-B/V). (7) 
xeS"- 1 iv Taxes'- 1 

To estimate S2, we will use the following notation 

M = max{V> 2 ri,max|X,| 2 }, E B = E B {x) = {i < N: \{Xi,x)\ > B}, m= sup \E B (x)\. 



By the definition of A m , we have for every x 6 S 



n-l 



B 2 \E B \< \{Xi,x)\ 2 < sup ^1(^,2;) | 2 < A 2 m , 

which yields B 2 m < and NS2(x) < A 2 m . Theorem [2] implies that for some absolute constant C > C2, 
with probability at least 1 — exp(—y/n) one has 



^m<C[M + ^ 2 mln 2 — ] and sup S 2 (x) < C ( — + ^ In 2 — ) . (M 



Now we choose the parameters. Let B = 2V2Cipln(5N/n). Then (J7J gives S 3 (x) < C-0 2 f < Cj^ for all 
x £ 5 f,l_1 and together with ([5J it yields that with probability at least 1 — exp(— y/n) one has 

sup (5 2 (x) + S 3 (x)) < C ((2M/N) + ^ 2 {m/N) ln 2 (2iV/m)) . 

It is easy to check that M > ip 2 m\n 2 (2N /m) on the set where ([8]) holds. Indeed, assume it is not so, 
thus M < V> 2 TOln 2 (27V/m). Then by (JS} we observe that B 2 < 2Ctp 2 \n 2 (2N/m), which implies 

m < 2Nexp(-5/V>V / 2(7) = 2n 2 /25iV. 
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By our hypothetical upper bound for M and since increases on [l,2N/e 2 ], we get 

V> 2 n < M < ij 2 (8n 2 /25N) ln 2 (5A/n), 

which is impossible. 
It follows that 

P f sup (S 2 (x) + S 3 (x)) < 3C(M/N)) > 1 - exp(-Vn). 
Combining this estimate with we get 

M\ ( 9 2 N 

sup S(x) <6 + 3C— > 1 - exp(- v / n) - exp f 



We now set 6* = C3?A 2 y/n/N, where C3 is a sufhciently large absolute positive constant so that ([5]) is satisfied. 
Then using boundedness condition with constant K we obtain 

P (sup Six) < (C 3 ip 2 + 3CK 2 ) y/^/N) > 1 - exp(-V^) - exp i-cn) > 1 - 2exp(-cV^), 

where c is a sufficiently small positive constant. It proves the desired estimate on the (I/3)-net. 

To pass from Af to the whole sphere note that S(x) can be written as \{Tx, x)\, where T is a self-adjoint 
operator on R™. Thus, writing for each x E S 1 ™ -1 , x = y + z with y g Af and \z\ < 1/3, we get 

||T||= sup \(Tx,x)\ < sup|(Ty, 2/ )| + ^sup|T 2/ |+ sup \(Tz, z)\ < sup S(y) + J|T||, 
which implies the desired estimate on the whole sphere S 1 " -1 . □ 
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